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lim 

n-*» 


J  <jl*Vx^~^x^ ^  =  resu^ts  9"*ve  substantial  improvements  upon 


the  existing  results. 


AMS  1980  subject  classifications:  Primary  62F10;  secondary  62H99. 
Key  word 's  and  phrases:  Data-based,  density  estimator,  empirical 


distribution,  histogram. 


Unci  assifi  ed 


‘-.LCUHI  r  r  CL  AS'.IMI  *  I  ION  Li  I  full,  HAC.l  (*!,»„  II.,.  (■  , 


REPORT  DOCUMENTATION  PAGE 


•  .  REPORT  NUMBER 


SET-"4  *- 


am 


READ  INSTRUCTIONS 
ItEI  ORK  foMPLETl'W;  KORM 


J.  RECIPIENT'S  CAT  ALOG  NUMBER 


•nil  Subtitle) 

Almost  Sure  L^-Norm  Convergence  for  Data-Based 


*•  type  of  report  *  period  covered 


Histogram  Density  Estimates 


Irnhnteiil  Report  -  Sept  1987 

.  roof /)/ 1 


•  performing  org.  report  number 


»•  AUTHOR!  e) 


••  contract  or  grant  number^) 


L.  C.  Zhao,  P.  R.  Krishnaiah,  and  X.  R.  Chen 


F49620-85-C-0008 


•  •  PERFORMING  ORGANIZATION  name  and  aooress 


Center  for  Multivariate  Analysis 
515  Thackeray  Hall 

rsitv  of  Pittsburgh,  Pittsburgh,  PA  15260 


1  CONTROLLING  office  name  ano  address 


10.  PROGRAM  ELEMENT.  PROJECT  TASK 
AREA  A  WORK  UNIT  NUMBERS 


LUCQf  3504  fi-T 


Air  Force  Of  fide  of  Scientific  Research 
Department  of  the  Air  Force 
Bolling  Air  Force  Base.  DC  20332 


•  4.  MONITORING  ACEnCY  NAME  I  AOORESSfff  dllletenl  Itoin  Com, oiling  Ollic*) 


U.  REPORT  DATE 

-SttHUliiihtir  1987  f^Oc 

is.  number  of  page! 

_ 26 _ 

SECURITY  CLASS,  (ot  iht a  tfpoft ) 


CO 


Unclassified 


is*,  declassification/ down  grading' 
schedule 


l«.  DISTRIBUTION  STATEMENT  <b I  (A la  Report) 


Approved  for  public  release;  distribution  unlimited 


<T.  DISTRIBUTION  STATEMENT  (ol  the  ebetrect  entered  In  Bloeb  JO,  II  dllletenl  from  Report) 


hi  :tu'.]T a. 


SUPPLEMENTARY  NOTES 


’*  KEY  BOROS  ( Continue  an  teveree  tide  ttne teeeery  end  Identity  by  bloeb  number)  ' 

<1  w 

Key  words  and  phrases:^  DataWbased,  density  estimator,'  empirical 

distribution,'  histogram. 


20  ABSTRACT  fCwiUnuo  on  ravmram  old o  If  nocotiory  ond  Idmntlly  by  block  numbot) 


DO  i^,  1473 


Unclassified 


SECURITY  CLASSIFICATION  OF  THIS  PAGE  (When  Dote  Entered) 


— ^  Let  iv,v.*n  be  1.1. d.  samples  drawn  from  a  d-djmensional  distribution 

with  density  f.  Partition  the  space  Into  a  union  of  disjoint  intervals 
{1^-  lU,X1,...,Xn)}  with  the  form  \ ^  ^  ■  (x(1\...  ,x^:  -  »  <  a^.^x*1 

<  b^<  *,  1  -  l»...»d|.  Define  the  data-based  histogram  estimate  of  f(x) 


based  on  this  partition  as 


£t'  f^(x)  =  num^er  *l,,v***n 

4  n  times  the  volume  of  1^,  for  x  €  I^»  l  *  1»2,... 

For  given  constant  r  >  1  we  obtain  the  sufficient  condition  for 

11m  f  (if  (xl-f(x)!^dx  3  0.  The  results  give  substantial  improvements  upon 

rv>«  J  n'  ' 

•V  J  /  /'  /  v 


the  existing  results. 


/  / 

•  •  '  L  r<  ■  '  '  ‘ 


/  1 


r  , 


,,  4 


; '  A  ' 
/ 


§11 

III 

SI 


1.  INTRODUCTION  AND  SUMMARY 


Suppose  that  Xj,...,Xn  are  i.i.d.  samples  of  a  d-dimensional 

random  vector  X.  Throughout  this  paper,  we  shall  denote  by  F  the 

distribution  of  X,  f  the  probability  density  function  of  X, 

Xn  =  (X, . X  ),  and  F  the  empirical  distribution  of  Xn. 

1  n  n 

Let  f  =  f  (x)  =  f  (x;Xn)  be  an  estimate  of  f  based  on  Xn. 
n  n  n 

For  any  constant  r  >  1,  define 


mnr  H  H  Jl  fn(x)-f  (x)  I  r<*x.  (1) 

Here  and  in  the  sequel,  J  means  J  ^  m^f,  to  be  called  the  Lr-norm  of 


fn  '  f’  is  a  much  "  studied  criterion  in  evaluating  the  performance  of  a 

density  estimator.  Quite  a  number  of  works  have  been  done  on  the 

problem  of  convergence  (to  zero)  of  mnr  as  the  sample  size  n  tends  to 

infinity.  We  say  that  f  is  a  L  -norm  consistent  estimator  of  f  if 

nr 

m  ->-0  as  n  in  some  sense, 
nr 

For  the  kernel  estimator 

fn(x)  =  (nhj)-1  j  KOTVx,)), 


where  the  kernel  is  assumed  to  be  a  probability  density,  Devroye  [8] 


proved  that  the  necessary  and  sufficient  conditions  for 

lim  mnj  =0,  a.s„ 
n-*» 


(2) 


n  For 
UI 


d  '  - 

are  that  hn  -*•  0,  nhn  -*■  ®.  Bai  and  Chen  [3]  solved  the  general  case  of 

r  >  1,  proving  that  the  necessary  and  sufficient  conditions  for  : , 
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% 
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lim  rn  „  =  0,  a.s.  for  some  r  >  1 
nr  — 

n-x=° 


are  that 


hn  ■>  0,  nhd  ->  jfr(x)dx  <  «.  Jl<r(u)du  <  <*>. 


In  the  case  of  kn-nearest  neighbor  estimator  proposed  by  Loftsgarden 


and  Quesenberry  [10],  Zhao  Vue  [12]  proved  that  a  sufficient  condition 


for  (3)  in  case  of  r  >  1  is  that 


kn/n  -*■  0,  kn/logn  -*  «,  jfr(x)dx 


while  the  first  and  last  in  (4),  and  also  that  kn  -*■  °°,  are  necessary  for 


the  truth  of  (3)  (r  >  1). 


Another  important  type  of  density-estimator  is  the  histogram  - 


ordinary  histogram  and  data-based  histogram.  In  ordinary  histogram. 


the  partitioning  of  range  space  of  X,R  ,  is  done  prior  to  the  drawing 
of  samples  Xn.  For  this  case,  Abou-Jaoude  [1],  [2]  (see  also 
Devroye  and  Gyorfi  [9],  pp. 19-23)  obtained  the  necessary  and  sufficient 
conditions  (imposed  on  the  partition)  for  the  truth  of  (2).  Chen  and 
Zhao  [7]  solved  the  general  case  of  r  >  1,  for  the  particular  partition 


Rd  =  I  I  {x  =  (x^ ,x^):a.  +  k.h  <  x^  <  a.  +  (k.+l)h  , 
i,  W  l,  =_oo  l  in-  i  i  '  n’ 


k  k 

fS.  j  >  •  O  O  f  IS  J 


1  <  i  <  d>. 


Data-based  histogram  differs  from  the  ordinary  one  in  that  the 


partition  of  space  R  defining  the  density  estimate  depends  on  the 


observations  Xn.  Thus,  after  obtaining  Xn,  we  make  a  partition  of 


$n  =  {I(l,Xn)U  =  1,2,...} 


Q  I U,Xn)  =  Rd,  I(j,Xn)  n  I(k,Xn)  =0  when  j  t  k). 


In  this  paper  we  consider  only  the  case  that  I(£,xn),  £  =  1,2,..., 

are  intervals  in  Rd  of  the  form 

»^i *^2 * *^^d *^d '  *  ~  00  — ^i  <  ^-j  —  °°»  1  <  i  <  d. 

For  each  x  e  Rd,  denote  by  In(x)  the  unique  interval  in  $n  containing  x, 

and  by  x(In(x))the  Lebesque  measure  of  In(x).  The  data-based  histogram 
estimate  fn»  based  on  the  partition  i s  defined  by 

fn(x)  s  Fn(In(x))/x(In(x)).  (5) 

For  this  estimate,  the  problem  of  L^-norm  consistency  is  much  more 

complicated  as  compared  with  the  ordinary  histogram  case.  To  begin  with, 
for  each  positive  integer  n  and  positive  constant  t,  denote  by  Cnt 

the  number  of  intervals  in  $n  =  {I(£,Xn)>  fulfiling  the  condition 

I (£,Xn)  n  {x  =  (x(1),...,x(d))  :  |x(i>l  <  t,  1  <  i  <  d)  f  0 

and  denote  by  D(A)  the  diameter  for  any  set  A «=  Rd.  Chen  and  Rubin 
[4]  proved  that 

lim  mnl  =  0,  in  probability  (6) 

n-+<*> 

under  three  conditions,  two  of  them  are: 

lim  D(In(x))  =  0,  in  probability,  for  x  e  ff1,  a.e.x  (7) 

|>+o° 


lim  C  ./n  =  0,  d  =  1; 
n  nt 

lim  C^/Zn  =  0,  d  >  1. 
0+0° 


in  probability,  for  any  t  >  0 


while  the  third  one  is  of  a  rather  complicated  nature.  Chen  and  Zhao 
[6]  studied  the  strong  consistency  for  the  case  of  general  d,  proving 
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v‘ 


3 
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the  truth  of  (2)  under  the  conditions: 

Tim  D( I  (x))  =  0,  a.s.,  for  x  e  Rd,  a.e.x  (7*) 

n+oo  n 

lim  C  logn/n  =  0,  a.s.,  for  any  t  >  0  (8*) 

n-v<» 

while  Chen  and  Wang  [5]  obtained  analogous  result  for  this  problem. 

By  comparing  (8)  and  (8*)  we  see  that  although  in  case  d  >  1  (8*) 
is  an  improvement  of  (8),  but  in  case  d  =  1,  in  achieving  strong 

Comparing  the  above  two  results,  we  see  that  in  case  d  >  1  we  have 
not  only  made  the  improvement  by  establishing  a.s.  convergence  instead  of 
convergence  in  probability,  but  also  succeeded  in  some  sense  in  weakening 
the  conditions  required,  since  (8*)  requires  a  lower  rate  of  convergence 
to  zero  than  (8)  -  Of  course,  strictly  speaking,  (8)  and  (8*)  are 
mutually  exclusive.  In  case  d  =  1,  in  achieving  strong  consistency,  we 
pay  a  price  by  requiring  that  cnt  is  of  the  order  0(n/logn)  instead  of 
0(n) .  Motivated  by  the  works  of  Devroye  [8],  Bai  and  Chen  [3]  and  Chen 
and  Zhao  [7],  we  expect  that  the  order  0(n)  should  be  sufficient.  In 
section  3,  we  shall  prove  that  this  is  indeed  the  case: 

THEOREM  1.  Suppose  that  f  is  defined  by  (5),  then  (2)  is  true 
if  (7*)  and  the  following  condition  (9)  are  both  true: 

lim  Cnt/n  =  0,  a.s.  for  any  t  >  0  (9) 

n-H» 

The  general  case  of  r  >  1  is  considered  in  section  4.  To  formulate 
our  result,  for  any  interval  I  =  [a^,b^)X...X[acj,b(j)  belonging  to  the 

partition  *  ,  write  a(I)  for  ).  Also,  for  any  t  >  0,  write 

=  {x  =  (x*1) . x^Mx^l  ^t,  i  =  1 ,. . . ,d}. 


,  .V 


THEOREM  2.  Suppose  that  f  is  defined  by  (5),  then  (3)  is  true 
if  the  following  three  conditions  are  satisfied: 

Jfr(x)dx  <  °°  (10) 

sup{D(  I ) :  I  e  *n,  I  n  Q  1  0  }  0  a,s.  for  any  t  >  0,  (11) 

n  v 

n(,1"f  a(I))d  -  •»,  a.s.  (12) 

The  basic  tool  in  our  argument  is  an  inequality  establishing 
the  exponential  bound  for  the  deviation  between  theoretical  and 
empirical  distributions  over  a  class  of  partitions  of  The 
inequality  is  of  independent  interest  and  Is  the  subject  of  section  2. 


- t~t  t  i  itti. 
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2.  AN  INEQUALITY 

Suppose  that  p  Is  a  probability  measure  on  Bd  -  the  0-  field  of 
all  Borel  sets  in  Rd.  Let  Xj,...Xn,..0  be  i.i.d.  random  vectors  with 

a  common  probability  distribution  p,  and  pn  be  the  empirical  distribu- 

tion  of  $ •  •  • 

We  call  «  =  { Aj , . . . »Ak>  a  partition  of  Rd,  if  A1 .  Ak  are 

disjoint  intervals  of  the  form 

[®j»^j)X««*X [a^j jbj)  t  -  °°  <_  a  ^  <  b^  <  °° ,  i  —  1 , 0 ,  „  ,d 

and  A.  =  Rd.  For  fixed  positive  integer  k,  denote  by  F  =  F.  the 
i  =1  k 

collection  of  all  such  partitions,  and  define 

D»  =  Dn(Xn)  5  sup(IA  e  ,|pn(A)-v(A)|  :  $  e  F} 

It  can  easily  be  seen  that  there  exists  a  countable  subset  {$>.. ,  i  =  l,  2,...} 

CF.  such  that  Dn  =  sup{tAet  |„n(A)  -  u<A)| :  1  =  1,2,...},  and  {♦  >  is  indepen- 
n 

dent  of  X  .  This  shows  that  is  a  random  variable.  We  are  now  going 

to  establish  the  following  exponential  bound  for  D  : 

n 

THEOREM  A.  Riven  e  e(0,l),  we  have 

P(Dn  >  e)  <  6  exp(-ne22”9) 

when  n  >  max(k,1001og6/e2),  and  (ii)log(.2jp)  <  e22"9(d+l)"1. 

In  proving  the  theorem,  we  borrow  some  idea  from  a  work  of  Vapnik 

and  Chervonenkis  [11],  We  also  note  that  Theorem  A  extends  a  work 

of  Devroye  [8],  which  we  formulate  below  as  a  lermia: 

LEWA  1  (Devroye  [8])  Suppose  that  IT  «  1^  ,  A.  e 


1  =  l,...,k,  and  A.  fl  A.  =  0  when  i  f  j.  Then  for  given  e  >  0 

*  wl 

we  have 

k  n  a 

p(  1  | u_( A . )-m(A. ) |  >  e)  <  3exp(-ne  /25),  when  k/n  <  e  /20 

i=l  n  i  1  -  - 

The  following  simple  fact  is  also  needed  in  the  proof: 

LEMMA  2.  Let  i  =  l,...,k,  be  positive  numbers. 

Write  a  =  xi ,  b  =  .  We  have 


k 

n 

i=l 


> 


and  the  equality  holds  if  and  only  if  qj  =  ...  =  q^. 

The  proof  Is  easy  and  therefore  omitted. 

Proof  of  the  Theorem. 

Write  ^  =  (xn+l,*0*,W*  ^  *  ^Xlt**°,X2n^  %  =  the 

empirical  measure  of  X'nL  and 


Dn(X"-t>  *  lAe>n<ft>-‘1<''>i 

Dn<x(")-‘,>  *  W“n<A>-“(AH 
6n<x2"-‘>  "  W“n<A>-“>>! 

Gn  5  Gn<x2n)  '  suPlSn(X2l'.f)’  :*  e  F) 

Since  (Gn  >  f)  («„(♦,)  >  |OUi=l{Dn(V  >  elfKD^.)  <  e/2> 

and  {Dn($i):  i  =  1,2,...},  { D* (<5^ ) :  i  =  1,2,...}  are  independent,  it  is 
well  known  that 

P(Gp  >  f)  >  inf  P(D*(*.)  <  e/2)P(^i:i{Dn($.)  >  e) 

=  if  P(D*($.)  <  e/2)P(Dn  >  e) 
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Suppose  that  n  satisfies  the  conditions  Indicated  In  Theorem  A,  then 
2 

k/n  <  e  /80,  and  by  Lemma  1  we  have,  simultaneously  for  all  Xn: 

P(Dn(X^n^$i)  >_  e/2 1 XP)  <  3exp(-ne2/100)  <  1/2,  1  =  1,2,... 

Therefore,  PtD*^)  <  e/2,  1  =  1,2 .  and 

P(Gn  >  -^(Dn  >  e)  03) 

From  (13),  it  is  seen  that  the  proof  of  Theorem  A  reduces  to  the  problem 
of  finding  an  upper  bound  for  P(Gn  >  e/2).  For  this  purpose,  denote  by  T 
a  permutation  )  of  (1,2, . . . ,2n) ,  so  that  TX2n  =  (X.  ,X. 

Further,  denote  by  and  the  empirical  measures  generated  by 

(X.  , « • • »X ,  )  and  (X.  ,...,X.  ),  respectively.  Then  It  Is  readily 

Jn+1  J2n 


seen  that 

P(Gn  >  f)  ■ 

lR2nd  TWT  j 

\  -  vJT)‘<A>l 

< 

f  2  1 

Jp2nd  (2nJ!  i 

f  It«eFUe*l,1nT,(A)‘1'2n(A,l  ’  f,dp  (14> 

where  the  summation  E  Is  taken  over  all  (2n ) !  permutations  of 

T 

(1 ,2 ,2n) ,  and  P  = 

Now  fix  X  ,  and  denote  by  U  the  set  with  elements  X,  ...,X„  . 

1 ,  2n 

Each  *  a  F  Induces  a  partition  of  the  set  U.  Denote  by  m  (U)  the 

n 

number  of  different  partitions  Induced  by  all  <t>  e  F.  we  have 

m„(U)  <  (2n^')d  £  <k3_n,)d  i  (^jV  <  Cf  )M  (15)  ! 


Let  F  be  a  subset  of  F  having  mn ( U )  members,  such  that  if 
$  e  F  ,  i  =  1,2  and  f  $2»  then  and  4>2  1"nduce  dl’f,ferent  partitions 


of  U.  We  have 


T5nTT  j  I{«F  Ail^T)(fl)-“2n(A)l  >f) 

'wr^rfi{Au^T)(A)-wA)i  >!* 

‘TOT*  «F*  I(ALi“iT,(A>-“2n(A)l  »f> 

1  Jf*  W!T  f  I{AL|u"T)(A>'“2n(A)l  >f! 

ItALl"»T>(A)'"a'(A>l  >f>  (16) 

Fix  <5  =  (A1,...,Ak)  e  F.  Denote  by  Y1»°**>Yn  a  random  sample  taken  from 

U  without  replacement,  and  (Z^ ,  i  >_1)  be  a  sequence  of  random  samples 

taken  from  u  with  replacement.  Write  P(-)  =  P(*|X2n),  E(*) 

=  E( •  |X2n),  and 

=  u2n^l^  ’  ~  ^  ~  ^***®*^ 


k  n 


k  n 


V  =  J  IK  l  KY.eA  )-nP,|,  W  =  III  I(Z.€A  )-np.| 
n  £= 1  i=l  1  £  41  n  £=1  i=l  1  £  z 


Then  we  have 


WTTf  HAHu'T)(A).„2n(A)|  >f> 
UnU  I(V,*At)  -  P*l  >f)>  ■  P(Vn  >  en/4) 


Now  we  proceed  to  show  that 


N 

K. 


E{exp(tVn)}  <_  (4iTe^^n/k)*<^E{exp(tWn) } 


for  any  t  >  0„  In  fact. 


E  exp(tW  ) 


^'-ni;-:-.!nk;  pini---pk"k  exp{t 


where  the  summation  z1  is  taken  over  all  integer-valued  vectors 
(n1,..a,nk)  satisfying 


n.  >  0,...,n.  >  0  and  £  n„  =  n. 
l  -  k  -  1=1  c 

In  the  same  way,  we  have 


E{exp(tVn)  =■  =:'(„}). •■(^)(2n)'Wt  J  J  n^-np^| } 


1=1 
k 


■  z'c<nl . "k’ipTT^  pl1-pkk  eXp{tJ1|n^'np^1  J 


Here,  as  usual ,  we  put  (^)  =  0  for  m  >  n.  Also, 


.-n 


J) 


(I) 


(19) 


(20) 


(21) 


where  IT  is  taken  over  all  j's  satisfying  N.  =  n.  and  H  is  taken 
(I)  J  J  (II) 


over  all  j's  satisfying  0  <  n.  <  N..  Using  Stirling's  formula 

J  J 

^  nne”n  <  n!  <  „V"+1/(12n) 


and  the 


fact  that  I  n.  =  n,  we 

j=l  J 


C ( n^ ).,<  ,n^ ) 


N.-n.  -(N.-n  )-l/2 
j  J  J 


<  2-n-l/2en*l/12(2,,k/2  (k/i2.  ,  „  ft"*,  '"j 

j=l  J  (II)  Nj 

<  2-n-1/2(2„)k/2e<ktl>''12  „  ,N,rnj,-(Nr"j)  n  rfT. 

(III)(  N“  j=l  3 


n  and  the  summation  z  appearing  below,  are  taken  over  all 
(HI)*  (III) 

j  1  s  satisfying  O^n.  <  N..  Putting  q.  =  (N.-n.)/N.,  x.  =  N.  ir 

J  J  J  J  J  J  J  J 


Lemma  2 ,  we  get 


a  a  z  x.  =  z  N.  <  2n, 
”  (III)  3  (III)  3 


Z  x,q.  =  z  (N.-n.)  =  V  (N.-n.)  =  n, 
(III)  J  J  (III)  3  3  jk  3  3 


„  N  .-n.  N  .-n .  „  x  .q .  . 

11  (  J  J  J  =  n  f h/a xb  -n 

(III)(  N~)  (III)  qj  -(b/a)  ^2  * 


On  the  other  hand. 


n  ^  <  (7  l  N  )k/2  =  (2n/k)* 
j=l  J  ”  K  j=l  J 


By  (22)  -  (24), 


C(n1, —  ,nk)  <  2"1/2e(k+1)/12(4Ttn/k)k/2  <  (47rn/k)k/2ek/12, 

and  (19)  follows  from  (20),  (21)  and  (25). 

Let  N  be  a  Poisson  random  variable,  E(N)  =  n,  and  N^Yj,...^ 

N 

Zj.Zg,...)  are  independent.  Since  l  I(Z.  e  A„),  i  =  l,...,k,  a 


independent  Poisson  variables  with  mean  np^,...,np^  respectively,  it 
follows  from  (19)  that  for  any  t  > 0, 

P{Vn  >  ne/4) 

<  P{ (N-n)  >  ne/8}  +  e"tne/4E{ et V°I ( | N-n |  <  ne/8)} 

=  P{  | N-n j  >  ne/8}  +  e‘tne/4EetVn  P{  |  N-n|  <_  ne/8} 

£  P{  | N-n  1  >  ne/8}  +  (4ire1/6n/k)k/2e“tne/4EetW|rip{  j  N-n]  <  ne/8} 

=  P{  | N-n |  >  ne/8}  +  (4Tre1/6n/k)k/2e"tne/4E{etWni(  |N-n|  <  ne/8}. 

From  the  independence  mentioned  above  and 
tu  k  N 

etWnI( | N-n |  <  ne/8)  <  exp{t  J  j  J  I(Z.eA„)  -  nPJ  +  tne/8}, 

£= 1  i=l  1  4  4 

it  follows  that 

P{Vn  >  ne/4} 

<_  P{  |  N-n  1  >  ne/8} 
k  N 

+  (4Tre1/6n/k)k/2e  -tn*Bn  E{exp(t|  l  I(Z.sAJ  -  nPJ)}.  (26 

1=  1  i=l  1  4  4 

Now  suppose  that  V  is  a  Poisson  variable  and  EV  =  a.  From 
e_t  +  t  _<  et  -  t  for  t  >  0,  it  follows  that 

E(etlv-xl)<E(et(V-l)*e-t(V-x>> 

=  exp{x(et-l-  t)}  +  exp{A(e-t-l+t)}  <_ 2exp{x(et-l-t)}. 

So  we  have 

P{  | V-X |  >_  Xe}  E{exp(t|V-X|-tXe)} 


<_2exp{-tXe  +  x^-l-t)}. 

Take  t  =  log(l+e),  we  get 

P{ }  V-A  |  >.  Ae}  <_2exp{x(e-  ( l+e)log  (1+e) ) } 

<  2exp{-Xe2/(2+2e)>  <  2exp(-Xe2/4) 

for  ee(0,l).  Repeat  this  argument  and  take  t  =  log(l+e/8),  by  (26) 
we  have 

P{  Vn  >  n  e/4} 

<  2exp(-ne2/256)  +  (4Tre1/6n/k)k/2e'tnc/8  n  {2exp(nP„(et-l-t))} 

1=1  L 

<  2exp(-ne2/256)  +  (4ire1/6n/k)k/2  2kexp{n(et-l-t-te/8)} 

<  2exp(-n£2/256)  +  (16Tre1/6n/k)k/2exp(-ne2/256) 

<  3(167re1/6n/k)k/2  exp(-  ne2/256).  (27 

From  (14)  -  (18)  and  (27),  it  follows  that 

P{Gn  >  e/2}  <  3(3en/k)kd(16ire1/6n/k)k/2exp(-ne2/256) 

=  3  exp{-ne2/256  +  kd  log  (3en/k)  +  ^log(16ire1/,6n/k)}. 

1/6  2 

Under  the  conditions  of  Theorem  A,  n/k  >  16  ei/0/(9e  )  and 
k(d+l)log(3en/k)  <  ne2/29.  Hence, 

P{Gn  >  e/2}  <3  exp{-ne2/256  +  k(d+l )log (3en/k) } 

<_  3  exp{-ne2/29}. 

From  this  and  (13),  Theorem  A  follows. 


3.  PROOF  OF  THEOREM  1 


Def i ne 

Vx)  =  h  U)fMdu/x(lU)),  (28) 

n'  '  1 

It  is  enough  to  show  that  for  any  t  >  0, 

ilH  Iq  lf(x)  “  Vx)ld*  =  0  a.s.  (29) 

and 

il!  /Qt lfn^x>  ‘  Vx)ldx  =  0  a‘s*  (3°) 

For  any  e  >  0,  we  can  find  a  function  g(x)  >_  0  which  is  continuous 
on  Rd  and  has  a  bounded  support,  such  that  J|f-g|dx  <  e.  Define 

gn(x)  =  h  (X)9(u)du/x(ln(x)). 

Then 

1q  lf-fnldx  1  Jlf-g|dx+flV9nldx+/q  iVgldx 

<  2j|f-g|dx+(Q  lgn-g|dx 

t 

<  2e  +  Jqjgn-g!dx.  (31) 

By  (7*),  there  exists  a  set  BQc:(Rd)”  such  that  P(BQ)  =  0  and  for 

uA  ( ,X^ » . . . )  e  B0  we  have  lim^^  D(In(x))  =  0  for  x  e  Rd,  a.e.A,  and 

in  turn  it  follows  that  lirri  g„(x)  =  g(x)  for  x  €  Rd,  a.e.x. 

ir*00  n 

By  the  dominated  convergence  theorem, 

^Jq  0n(x)-g(x)  jdx  =  0  a.s.,  (32) 

t 

and  (29)  follows  from  (31)  and  (32), 


From  (9)  it  can  be  shown  that  there  exists  a  sequence  (5n)  of 

positive  numbers  such  that  lim  5=0  and 

n-*-°°  n 

^Cnt/[nsn]  =  0  a's” 

where  [n<$n]  denotes  the  integer  part  of  n6n.  For  any  ee(0,l),  there  exists 
a  set  (Rd)“  such  that  P(B-_g/2)  >  1  -  e/2  and 

cnt/[n6n]  ■  0  uniformly  for  (X^,...)  e 

So  there  exists  a  positive  integer  N  such  that 

Cnt  <  [n5n]»  for  nlN  and  (X^X^...)  e  Bj  e/2 

Now  we  recall  $n  =  $n(Xn)  =  UU,Xn),  l  =  1,2,...}  is  the 

partition  of  Rd  which  defines  the  data-based  histogram  f  .  It  is  easy 

to  see  that  we  can  find  k  <  3d  C  *  and  $  €  F,  such  that 

—  nt  k 

{I  :  I  e  *,I  fl  Qt  f  0}  =  (I  :  I  e  *n,i  fl  Qt  f  0}. 

Hence,  for  (X^.X^,...)  e  Bj^^  n  1  N  and  k  =  3d[n6n],  we  have 

Jqlfn(x>  -  Vx>ldx  1  Eie»  .WQ.rtlfnd)  -  F<!>! 

v  n  x 

i«Fk  EAeJFn<A>  -  F<A>I  iV  («) 

Since  k/n  =  3d[n6n]/n  <_  3dsn  -*•  0  as  n  -*■  ®,  from  Theorem  A,  we  have 

ii!  -  0  a.s.  (34) 


By  (33)  and  (34),  there  exists  a  set  B,  C(Rd)“  such  that  B, 

l*e  l~i 


l-e/2 , 


P(B,  )  >  1  -  e  and 


r-jm  ’iw 


W*J  F''F  I  WW  '  1H  I'PMW^  >  m  IP'^W 
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4.  PROOF  OF  THEOREM  2 


Define  f p ( x )  as  before  by  (28).  Find  a  nonnegative  function  g, 

d 


continuous  everywhere  on  R  and  with  a  bounded  support,  such  that 
J | f-g | rdx  <  er.  Put 

V  '  'i(X)'’  '~n 

Then 

(J|f-fn|rdx)1/r  <  (J|f-g|rdx)1/r  +  (J|fn-gn|rdx)1/r  +  (j[gn-g|rdx)1/r 


g„(x)  =  It  /v\9(u)du/x(In(x)). 


<  2(J|f-g[rdx)1/r  +  (J|gn-g|rdx)1/r 
<  2e  +  (J|gn-g|rdx)1/r. 


(35) 


By  (11),  for  any  x  e  R  ,  we  have 


D(In(x))  -  0  a.s. 


There  exists  a  set  B0<Z(Rd)°°  such  that  P(BQ)  =  0  and  for 

*»  =  (X1,X2,...)  e  Bq  we  have  11m  D(I  (x))  =0  for  x  e  Rd,  a.e.x, 

and  in  turn  it  follows  that  lim_  g_(x)  =  g(x)  for  x  e  Rd,  a.e.x.  By 

» II 

the  dominated  convergence  theorem. 


lim 

n-*-°° 


JlV9l 

By  (35)  and  (36),  we  have 


dx  *  0  for  w  e  BQ. 


(36) 


1  im 


£  J|f(x)-fn(x)|dx  =  0  a.s. 


(37) 


Now  we  proceed  to  prove  that 


Jivf„r<“  -  i  !Fnd£)  -  pdpr/xdp 


r-1 


0,  a.s. 


(38) 


•r  n 


*j**ji' »  'j  •^^*'**  *•'*»**■  */ 


>  -■>  .% 
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Put  H  =  inf.  a(I).  Since  nHa  -*■  ®  a.s.,  it  can  be  shown  that  there 


exists  a  sequence  C  ®  such  that  Tim  nH  /C  =  <*>  a.s..  Without 

n  n-x»  n 


loss  of  generality,  we  can  assume  that  Cd/n  -*■  0.  Take  h  =  h  =  C  /n1//d, 
3  J  n'  n  n  * 


then  hn  -*■  0,  nhn  -*■  ®  and  H/hn  °°,  a.s. 


Construct  a  partition  of  R  into  disjoint  finite  intervals,  say 


t'n  =  »...}»  where  Ams  are  all  cubes  with  the  same  edge  length  h. 


Defi ne 


5n(x)  =  Fn(Am)/h  for  x  e  Am,  m  =  1,2,... 


Cn(x)  *  F(Am)/h  for  x  e  a  ,  m  =  1,2,... 


By  the  theorem  of  [7], 


Jl5n(*)-f(x)!rdx  =  0.  a.s. 


An  argument  similar  to  that  leading  to  (37)  gives 


i:  {un<x)-f(x)irdx  =  o. 


So  we  have 


JU„(x)-'E„(x)|rdx  .  I  |F„(dra)-FUm)r/(hd)r-1 


A  6¥ 

m  n 


-*■  0  a.s.  as  n  -*■ 


For  I^e^,  denote  by  .  ...H^  the  lengths  of  the  edges  of  1^,  and 


write 


V  (m  :  an,  e  V  An  =V- 


=  «”  :  An  6  V  An  n  h  *  *>  Ajx'i  *  0)' 


’r'vW,  /i  >\r 


>->'v  ji  v  jaBCvs>  • 


Since  H/hp  -*■  ®  a.s.,  we  can  find  B*^.(R  )  such  that  P(B*)  = 
and  H/h^  ®  for  we"  B*.  In  the  sequel  we  always  keep  w  e  B 

Thus,  for  n  large,  H..£  >  2h  for  all  i  and  l.  We  have 

d  d 

MHu/h  -2)<  #(M£)  <  MHu/h), 

and 

d  d 

#(M„)  <  n  (H.  Jh  +2) -  n  (H. p/h  -2) 

4  i=l  141  i  =1  141 

.  d  d 

<  x(I  Jh_a{  ti  (l+2h/H.J  -  n  (l-2h/H.  „)} 

41  i  =1  141  i  =1  u 

1  X(I£)h"d{(U2h/H)d  -  (l-2h/H)d } 


<  h-ax(I£)C(d)h/H, 


where 


C(d)  =  2d(2d"1+l). 


Now,  by  (41)  and  (39)  we  have 

i,£in(,<v>r-u;F"(^-F^,,rA!i/'1 

-  I  a-s- 

Vfn 

On  the  other  hand,  by  (42)  we  have 

pn  4  I  |Fn(  l.  (I£A  ))  -  F{  I.  (I  A  ))|r/x(I  )r-1 
n  l„e®  n  meM„  meM„  1 


L  (,(fi£),r  Jr,  -  FUA>lr/x<V' 


I  e$ 

/  n 


v»v  ,S 


(44) 


which  implies  that 


From  (46)  -  (48),  we  obtain 


By  (43)  and  (49),  we  have 

J|fn(x)  -  ?n(x)|rdx  =  l  |F  (I£)  -  F(I£)|r'/»(I£)r"1 
^  2r"^(pn+pn)  *  0  a.s. 

Thus,  (38)  is  proved,  and  Theorem  2  follows  from  (37)  and  (38) 


wv 


fc 

4 

s' 

V 

«>■ 


I 

1 
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